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End-to-End  Testing  for  Personnel  Systems 


Executive  Summary 


Introduction.  This  report  is  one  in  a  series  of  reports  that  the  Inspector  General,  DoD, 
is  issuing  in  accordance  with  an  informal  partnership  with  the  DoD  Chief  Information 
Officer  to  monitor  DoD  efforts  in  addressing  the  year  2000  computer  challenge.  For  a 
listing  of  audit  projects  addressing  the  issue,  see  the  year  2000  web  pages  on  the  IGnet 
at  http://www.ignet.gov. 

The  “DoD  Year  2000  Management  Plan,”  Appendix  I,  “Guidelines  to  Support  DoD 
Y2K  Operational  Readiness,”  assigns  responsibility  to  the  Principal  Staff  Assistants  for 
“ensuring  the  end-to-end  functional  process  flows  that  support  their  functional  area  are 
assessed  either  in  a  JS/CINC  [Joint  Staff/Commander  In  Chief]  Y2K  Op  Eval 
[Operational  Evaluation],  a  Service-sponsored  System  Integration  Test,  or  through  a 
Functional-Area  Y2K  End-to-End  Test.  ”  Appendix  I  also  states  that  the  Principal  Staff 
Assistants’  responsibilities  include  “planning,  executing,  and  evaluating  all  mission- 
critical  systems  not  otherwise  tested  and  for  ensuring  that  processes  that  fall  within  their 
purview  are  evaluated.”  The  Under  Secretary  of  Defense  for  Personnel  and  Readiness 
performs  those  functions  for  personnel. 

Objectives.  The  overall  objective  was  to  evaluate  the  effectiveness  of  the  planned 
year  2000  end-to-end  tests  for  personnel  systems.  Specifically,  we  reviewed  the  test 
plans  and  the  results  of  selected  test  events.  The  purpose  of  end-to-end  testing  is  to 
verify  that  the  test  of  interrelated  systems  supporting  an  organization  function  such  as 
personnel  transactions  operate  as  intended  in  an  operational  environment. 

Results.  The  Army,  Navy,  Air  Force,  and  Marine  Corps  have  been  conducting  end-to- 
end  testing  of  their  personnel  systems.  More  needs  to  be  done  to  provide  assurance  that 
personnel  systems  will  function  properly  in  the  year  2000.  Because  the  Navy  did  not 
begin  its  testing  until  September  1999,  a  risk  exists  that  the  Navy  will  not  complete  its 
end-to-end  testing  and  analysis  of  the  testing  of  its  systems  before  the  year  2000, 
although  the  Navy  remains  confident  that  the  testing  plan  can  be  executed. 

The  Air  Force  end-to-end  test  involving  the  Defense  Civilian  Personnel  Data  System 
was  not  as  rigorous  as  required  by  the  criteria  set  forth  in  “DoD  Year  2000 
Management  Plan.”  Specifically,  the  “DoD  Year  2000  Management  Plan”  states  that 
each  mission-critical  system  must  be  evaluated  at  least  once  in  either  a  functional  area 
year  2000  end-to-end  test  or  a  Service-sponsored  year  2000  system  integration  test. 

The  Air  Force  chose  to  use  test  data  that  it  saved  from  the  system  certification  test  of 


the  Defense  Civilian  Personnel  Data  System  for  its  end-to-end  testing.  As  a  result,  the 
end-to  end  test  as  conducted  did  not  provide  additional  assurance  that  year  2000  risk  for 
the  Defense  Civilian  Personnel  Data  System  had  been  reduced.  See  the  Finding  section 
of  the  report  for  a  discussion  of  the  audit  results. 

Summary  of  Recommendations.  Sufficient  risk  remains  to  warrant  additional  risk 
mitigation  measures  by  the  Navy.  To  provide  additional  assurance  that  the  year  2000 
risk  in  personnel  systems  is  reduced,  we  recommend  that  the  Deputy  Chief  of  Naval 
Personnel  require  that  interface  reexamination  procedures  be  performed  for  all  Navy 
mission-critical  personnel  systems  and  that  advanced  automated  scanning  tools  be  used 
to  examine  100  percent  of  the  application  software  for  those  systems. 

Because  of  the  remaining  risk  of  not  exercising  the  Defense  Civilian  Personnel  Data 
System  in  the  end-to  end  test,  the  effectiveness  of  the  contingency  plan  for  the  Defense 
Civilian  Personnel  Data  System  and  the  results  of  exercising  the  contingency  plan 
become  increasingly  important.  We  are  not  making  a  recommendation  to  the  Air  Force 
because  the  Inspector  General,  DoD,  has  announced  an  audit  of  the  effectiveness  of 
year  2000  operational  contingency  plans  for  DoD  personnel  systems,  which  will  include 
the  Defense  Civilian  Personnel  Data  System.  In  addition,  the  Air  Force  has  plans  to 
scan  the  application  software  in  its  mission-critical  personnel  and  finance  systems. 

Management  Comments.  The  Chief  of  Naval  Personnel,  Bureau  of  Naval  Personnel, 
concurred  and  stated  that  end-to-end  testing  started  September  4,  1999,  and  will  be 
completed  by  the  end  of  October.  The  Navy  also  stated  that  code  scanning  was  in 
process,  interface  testing  is  a  continuous  ongoing  process,  and  that  draft  contingency  of 
operations  plans  were  developed.  The  Navy  was  confident  that  the  personnel  systems 
will  operate  after  January  1,  2000.  The  Air  Force  disagreed  that  its  testing  was  not  as 
rigorous  as  required  by  the  DoD  Year  2000  Management  Plan.  A  discussion  of 
management  comments  is  in  the  Finding  section  of  the  report,  and  the  complete  text  is 
in  the  Management  Comments  section. 

Audit  Response.  The  Navy  comments  were  responsive.  Although  there  was  extensive 
Air  Force  testing,  the  Defense  Civilian  Personnel  Data  System  was  not  exercised  in  a 
separate,  higher  level  test  as  required  by  the  DoD  Year  2000  Management  Plan. 
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Background 


The  Year  2000  (Y2K)  Problem.  Computer  systems  have  typically  been 
designed  to  use  only  the  last  two  digits  for  the  year;  thus,  the  year  2000  is 
indistinguishable  from  the  year  1900.  As  a  consequence,  computers  and 
associated  software  that  use  dates  to  calculate,  compare,  or  sort  data  could 
generate  incorrect  results  when  working  with  years  after  1999.  The  potential 
for  computer  system  failure  after  the  year  1999  is  often  referred  to  as  the 
Y2K  problem. 

End-to-End  Testing.  The  end-to-end  process  is  a  complete  flow  of  data 
through  a  set  of  interconnected  systems  that  perform  a  core  business  process, 
function  or  mission.  Data  flow  begins  with  the  initial  input  (external  input)  to 
the  first  system  from  a  user  or  customer  and  ends  with  either  the  final  receipt  of 
information  in  the  final  destination  systems  or  receipt  of  output  by  the  user  or 
customer. 

Executive  Order.  Because  of  the  potential  failure  of  computers  to  function 
throughout  the  Government,  the  President  issued  Executive  Order  13073, 

“Year  2000  Conversion,”  February  4,  1998,  making  it  policy  that  Federal 
agencies  ensure  that  no  critical  Federal  program  experiences  disruption  because 
of  the  Y2K  problem.  The  order  requires  that  the  head  of  each  agency  ensure 
that  efforts  to  address  the  Y2K  problem  receive  the  highest  priority  attention  in 
the  agency. 

Public  Law.  Public  Law  105-261,  “National  Defense  Authorization  Act  for 
Fiscal  Year  1999,”  October  17,  1998,  Section  334(b)  directs  that  the  Secretary 
of  Defense  ensure  that  “all  mission-critical  systems  that  are  expected  to  be  used 
if  the  Armed  Forces  are  involved  in  a  conflict  in  a  major  theater  of  war  are 
tested  in  at  least  two  exercises.”  In  addition,  Section  334(d),  states: 

Alternative  Testing  Method.  In  the  case  of  an  information  technology 
or  national  security  system  for  which  a  simulated  Y2K  test  as  a  part  of 
a  military  exercise  described  in  subsection  (c)  is  not  feasible  or 
presents  undue  risk,  the  Secretary  of  Defense  shall  test  the  system 
using  a  functional  end-to-end  test  or  through  a  Defense  Major  Range 
and  Test  Facility  Base 

DoD  Year  2000  Management  Strategy.  In  his  role  as  the  DoD  Chief 
Information  Officer,  the  Senior  Civilian  Official,  Office  of  the  Assistant 
Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence), 
issued  the  “DoD  Year  2000  Management  Plan”  (DoD  Management  Plan)  in 
December  1998.  The  DoD  Management  Plan  requires  DoD  Components  to 
implement  a  five-phase  (awareness,  assessment,  renovation,  validation,  and 
implementation)  Y2K  management  process  to  be  completed  by  December  31, 
1998,  for  mission-critical  systems. 
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The  DoD  Management  Plan  provides  guidance  for  implementing  the  Deputy 
Secretary  of  Defense  Memorandum,  “Year  2000  Verification  of  National 
Security  Capabilities,”  August  24,  1998.  The  memorandum  requires  that  each 
Principal  Staff  Assistant  of  the  Office  of  the  Secretary  of  Defense  “verify  that 
all  functions  under  his  or  her  purview  will  continue  unaffected  by  Y2K  issues.” 
That  verification  was  to  be  performed  after  completion  of  the  five-phased 
management  approach  that  culminated  with  the  completion  of  the 
implementation  phase.  Further  verification  testing  was  to  be  conducted  during 
the  first  half  of  1999  and  was  to  be  planned  and  conducted  from  a  mission 
perspective,  rather  than  a  system  perspective,  to  increase  the  confidence  that  any 
errors  or  omissions  in  system  remediation  would  be  found. 

Objectives 


The  overall  objective  was  to  evaluate  the  effectiveness  of  the  planned  year  2000 
end-to-end  tests  for  personnel  systems.  Specifically,  we  reviewed  the  test  plans 
and  the  results  of  selected  test  events.  See  Appendix  A  for  a  discussion  of  the 
audit  scope,  methodology,  and  prior  audit  coverage. 
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End-to-End  Testing  of  Personnel  Systems 

The  Army,  Navy,  Air  Force,  and  Marine  Corps  have  been  conducting 
end-to-end  testing  of  their  personnel  systems.  However,  more  needs  to 
be  done  to  provide  assurance  that  personnel  systems  will  function 
properly  in  the  year  2000.  Because  the  Navy  did  not  begin  its  testing 
until  September  1999,  a  risk  exists  that  the  Navy  will  not  complete  its 
end-to-end  testing  and  analysis  of  the  testing  of  its  systems  before  the 
year  2000,  although  tire  Navy  remains  confident  that  the  testing  plan  can 
be  executed.  Also,  the  Air  Force  end-to-end  test  involving  the  Defense 
Civilian  Personnel  Data  System  was  not  as  rigorous  as  required  by  the 
criteria  set  forth  in  DoD  Management  Plan.  Instead,  the  Air  Force  chose 
to  use  test  data  that  it  saved  from  the  system  certification  test  of  the 
Defense  Civilian  Personnel  Data  System.  As  a  result,  the  end-to-end  test 
as  conducted  did  not  provide  additional  assurances  that  year  2000  risk 
for  the  Defense  Civilian  Personnel  Data  System  had  been  reduced. 

DoD  Guidance 


The  DoD  Management  Plan,  Appendix  I,  “Guidelines  to  Support  DoD  Y2K 
Operational  Readiness,”  assigns  responsibility  to  the  Principal  Staff  Assistants 
for  “ensuring  the  end-to-end  functional  process  flows  that  support  their 
functional  area  are  assessed  either  in  a  JS/CINC  [Joint  Staff/Commander  In 
Chief]  Y2K  Op  Eval  [Operational  Evaluation],  a  Service-sponsored  System 
Integration  Test,  or  through  a  Functional-Area  Y2K  End-to-End  Test.” 
Appendix  I  also  states  that  the  Principal  Staff  Assistants’  responsibilities  include 
“planning,  executing,  and  evaluating  all  mission-critical  systems  not  otherwise 
tested  and  for  ensuring  that  processes  that  fall  within  their  purview  are 
evaluated.” 

The  Under  Secretary  of  Defense  for  Personnel  and  Readiness  (USD[P&R]) 
performs  those  functions  for  the  personnel  functional  area.  As  the  Principal 
Staff  Assistant  for  personnel,  the  USD(P&R)  had  the  responsibility  to: 

•  provide  functional  end-to-end  test  plans; 

•  certify  that  the  test  plans  include  assessments  of  functional  risk, 
effects  of  Y2K  on  continuity  of  business  operations,  and  associated 
contingency  plans; 

•  ensure  that  all  test  plans  include  a  listing  of  all  mission-critical 
systems  to  be  involved  in  each  test;  and 

•  coordinate  each  test  plan  with  the  Military  Services  and  all  other 
pertinent  Principal  Staff  Assistants. 
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Personnel  Systems  Environment 


DoD  military  personnel,  manpower,  and  training  systems  are  primarily 
developed,  funded,  and  operated  by  the  Military  Services.  Each  Military  Service 
has  a  unique  set  of  military  personnel  requirements  that  involve  numerous 
systems  and  interfaces.  While  some  functional  cross-Service  activity  occurs, 
this  generally  happens  outside  of  the  information  technology  area.  As  a  result, 
each  Military  Service  needed  to  plan  and  conduct  its  own  end-to-end  test  for 
personnel  systems.  Also,  each  Military  Service  was  developing  its  own  test 
plans,  contingency  plans,  and  continuity  of  operation  plans.  Civilian  personnel 
functions  within  DoD  are  managed  through  a  single  personnel  system,  the 
Defense  Civilian  Personnel  Data  System  (DCPDS),  managed  by  the  Defense 
Civilian  Personnel  Management  Service,  which  contracts  with  the  Air  Force  to 
operate  the  legacy  and  interim  DCPDS. 


DoD  Personnel  Functions 

The  USD(P&R)  developed  the  Master  Plan,  April  5,  1999,  to  provide  policy 
oversight  and  to  coordinate  DoD-wide  initiatives  and  reporting  for  personnel 
systems  Y2K  functional  end-to-end  testing.  The  Master  Plan  was  augmented 
with  draft  test  plans  from  the  Army,  Navy,  Air  Force,  and  Marine  Corps.  The 
Master  Plan  focuses  on  identifying  the  test  threads  to  be  tested  and  the 
supporting  programs  and  interfaces  that  are  essential  for  successful  operation  of 
a  thread.  A  thread  is  a  group  of  systems  that  support  a  core  process  or  critical 
function.  The  Military  Services  are  required  to  fully  evaluate  all  functions  that 
the  USD(P&R)  designated  as  critical.  To  ensure  the  critical  functions  within  the 
personnel  area  were  tested  and  operational,  the  USD(P&R)  identified  the 
following  eight  personnel  functions  critical  to  support  the  soldier  during  wartime 
and  provided  a  brief  description. 

•  Access:  the  functional  process  for  bringing  civilians  into  the 
Military  Services. 

•  Mobilize:  the  activation  of  guard  and  reserve  members  to  active  duty 
status. 

•  Deploy:  the  personnel  activities  relating  to  the  movement  of 
personnel  from  their  normal  duty  station  to  the  site  of  an  operational 
location  where  they  are  needed. 

•  Locate:  the  Military  Services’  ability  to  find  individuals  based  on  a 
geographic  location  code. 

•  Pay:  the  ability  to  provide  personnel  data  to  the  pay  systems. 

•  Separate:  the  separation  of  a  member  from  the  Military  Service  for 
all  reasons. 
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•  Retire:  the  regular  and  disability  retirement  process  from  the 
Services. 

•  Casualty  Support:  the  activities  dealing  with  supporting  the  reporting 
and  tracking  of  casualties. 

The  Master  Plan  states  that  the  USD(P&R)  will  ensure  that  the  Military 
Services’  end-to-end  testing  efforts  incorporate  the  requirements  necessary  to 
conduct  the  test.  It  also  states  that  the  USD(P&R)  will  participate  in  verification 
of  the  test  and  test  results  of  the  Military  Services  functional  personnel 
end-to-end  tests. 

The  end-to-end  testing  of  personnel  systems  supporting  the  8  critical  functions 
was  limited  to  36  of  the  91  mission-critical  personnel  systems  that  the 
USD(P&R)  and  the  Military  Services  identified.  The  36  mission-critical 
systems  appeared  in  50  thinline  threads  in  the  8  mission-critical  personnel 
functions  that  needed  to  be  tested. 

Scheduling  the  Personnel  End-to-End  Testing 

The  Army,  Air  Force,  and  Marine  Corps  have  completed  their  portions  of  the 
actual  end-to-end  testing  of  personnel  systems.  They  were  evaluating  results 
and  waiting  for  the  Defense  Finance  and  Accounting  Service  to  coordinate  and 
complete  the  testing.  The  Navy  was  not  able  to  start  its  testing  until 
September  1999,  with  a  completion  date  of  October  17,  1999.  Several  factors 
led  to  the  delay  in  starting  the  testing.  The  memorandum  of  agreement  for 
system  certification  testing  requirements  between  the  Navy  and  the  Defense 
Information  Systems  Agency  was  signed  on  September  15,  1998.  The  Defense 
Information  Systems  Agency  was  unable  to  provide  the  necessary  platforms, 
manpower,  and  services  to  meet  the  original  test  requirements  identified  by  the 
Navy  for  its  mission-critical  systems  to  be  tested  in  accordance  with  the  DoD 
year  2000  system  certification  process.  Further  complicating  matters,  the 
Defense  Information  Systems  Agency  was  also  contemplating  moving  the  Navy 
systems  from  the  Defense  Megacenter  -  Chambersburg,  Pennsylvania,  to  the 
Defense  Megacenter  -  Columbus,  Ohio,  in  early  1999.  The  Navy  then  had  to 
refine  the  requirement  and  obtain  Defense  Information  Systems  Agency 
agreement  that  the  migration  of  personnel  would  not  occur  until  after  Y2K.  In 
the  agreement  for  system  certification  testing,  January  31,  1999,  was  established 
as  the  availability  date  of  hardware  and  personnel  needed  to  start  the  testing  and 
remediation  of  systems.  The  requirement  for  DoD  mandated  end-to-end  testing 
had  not  been  identified  when  the  agreement  was  signed.  During  the  remediation 
period,  the  DoD  identified  a  need  to  conduct  end-to-end  testing  for 
mission-critical  personnel  and  readiness  functions.  The  requirements  for  the 
end-to-end  testing  mirrored  the  hardware,  software,  and  personnel  requirements 
that  were  allocated  to  the  system  certification  testing.  A  memorandum  of 
agreement  for  the  end-to-end  testing  was  signed  between  the  Navy  and  the 
Defense  Information  Systems  Agency  on  April  27,  1999.  However,  the  end-to- 
end  testing  could  not  begin  until  the  Navy  had  completed  Y2K  system 
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certification  for  all  mission-critical  systems.  The  end-to-end  testing  began  in 
September  1999.  The  following  table  shows  start  and  completion  dates  for  the 
Military  Services’  personnel  end-to-end  testing. 


Military  Services  End-to-End  Personnel  Testing  Schedule 

Military  Service  _ Dates  of  Testing _ 


Start 


Army 
Navy 
Air  Force 
Marine  Corps 


June  1,  1999 
September  4,  1999 
May  1,  1999 
July  26,  1999 


Completion 

July  23,  1999 
October  17,  1999 
June  30,  1999 
August  20,  1999 


By  not  being  able  to  start  its  end-to-end  testing  until  September  1999,  the  Navy 
may  not  have  sufficient  time  to  complete  the  testing,  evaluate  the  test  results, 
resolve  Y2K  issues  that  might  be  encountered,  and  retest,  if  necessary.  As  the 
table  above  indicates,  the  actual  testing  for  the  Army  and  the  Air  Force  took  one 
to  two  months.  The  Army  and  Air  Force  planned  to  take  an  additional  month  or 
longer  to  evaluate  the  results  and  prepare  the  test  reports  for  their  end-to-end 
tests.  In  addition,  the  Air  Force  was  waiting  for  feedback  on  data  submitted  to 
the  Defense  Finance  and  Accounting  Service  for  the  portion  of  the  end-to-end 
test  related  to  pay.  The  Army  was  also  waiting  for  the  Defense  Finance  and 
Accounting  Service  to  finalize  details  on  how  they  will  exchange  data  in 
completing  the  end-to-end  test  on  the  pay  function.  The  Marine  Corps 
scheduled  additional  end-to-end  testing  of  the  mobilization  function. 


The  USD(P&R)  has  expressed  concerns  regarding  the  Navy  ability  to  complete 
end-to-end  testing  of  its  personnel  systems  within  the  timeframe  established. 

The  USD(P&R)  concerns  related  to  the  possible  lack  of  Navy  financial 
resources  to  accomplish  the  required  testing  and  lack  of  the  Bureau  of  Naval 
Personnel  system  documentation  and  institutional  knowledge.  Although  the 
Navy  was  confident  that  it  will  be  able  to  resolve  any  Y2K  errors  before  the 
start  of  year  2000,  its  timing  is  such  that  a  significant  amount  of  remaining  risk 
exists  to  warrant  placing  particular  importance  on  the  validity  of  the  contingency 
plans.  Additional  risk  mitigation  measures  are  needed  and  the  USD(P&R)  needs 
to  be  aware  of  the  results  of  the  actions  taken.  One  such  measure  would  be  an 
independent  review  of  the  contingency  plans  and  the  results  of  exercising  the 
contingency  plans  for  the  personnel  systems  in  the  Navy.  We  are  not  making  a 
recommendation  concerning  contingency  plans  because,  on  August  5,  1999,  the 
Inspector  General,  DoD,  announced  an  audit  of  “Year  2000  Contingency  Plans 
for  Personnel  Systems.”  The  objective  of  that  audit  is  to  evaluate  the 
effectiveness  of  the  year  2000  operational  contingency  plans  for  personnel 
systems.  The  review  should  determine  if  the  contingency  plans  detail  the 
procedures  necessary  to  restore  a  system  in  the  face  of  all  anticipated  and 
unanticipated  Y2K  disruptions  and  also  provide  for  continuing  operations  when 
support  from  a  single  system  or  group  of  closely  related  systems  is  disrupted. 
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Use  of  System  Level  Test  Results  for  End-to-End  Testing 

The  Air  Force  end-to-end  test  involving  the  DCPDS  was  not  as  rigorous  as 
required  by  the  criteria  set  forth  in  DoD  Management  Plan.  The  DoD 
Management  Plan  clearly  states  that  upon  completion  of  the  testing  of  the 
individual  systems  using  the  five-phase  process,  it  is  necessary  to  demonstrate 
the  Y2K  readiness  of  systems  in  an  integrated,  operational  environment 
Specifically,  with  few  exceptions,  all  mission-critical  systems  must  be  evaluated 
at  least  once  in  either  a  functional  area  Y2K  end-to-end  test  or  a  Service- 
sponsored  Y2K  System  Integration  Test.  The  DCPDS  did  complete  the 
system-level  test,  but  the  DCPDS  was  not  exercised  in  a  separate,  higher  level 
test  as  required  by  the  Defense  Management  Plan.  An  end-to-end  test  is 
conducted  by  inputting  data  into  the  system,  allowing  the  system  to  process  the 
data,  and  passing  the  data  to  other  systems  involved  in  the  end-to-end  testing. 
Instead  of  including  DCPDS  in  the  end-to-end  test,  the  Air  Force  chose  to  use 
test  data  that  it  saved  from  the  system  certification  test  of  the  DCPDS  for  its 
end-to-end  testing.  The  data  from  the  DCPDS  was  properly  passed  to  other 
systems;  therefore,  the  interfaces  between  those  systems  were  properly  tested. 
However,  the  Air  Force  did  not  retest  input  and  processing  by  the  DCPDS 
during  the  end-to-end  test. 

The  Air  Force  stated  that  there  were  several  advantages  to  using  the  saved  data. 
The  Air  Force  was  able  to  save  $2.3  million  by  eliminating  the  need  to  replicate 
the  DCPDS  hardware  for  the  end-to-end  test,  duplicate  the  operation  of  the 
DCPDS  just  as  it  is  used  in  the  field,  and  process  and  test  more  personnel 
transactions  affecting  pay.  After  self-certification  of  the  system,  the  Air  Force 
saved  the  data  and  transactions  from  the  system  test.  The  participation  of  the 
DCPDS  in  the  end-to-end  test  merely  consisted  of  the  DCPDS  passing  the  saved 
data  to  the  Defense  Civilian  Pay  System.  On  June  7,  1999,  the  contractor,  hired 
to  provide  independent  assessments  to  the  USD  (P&R)  of  the  end-to-end  test 
performed,  stated  that  because  DCPDS  “has  been  operational  for  a  long  time 
and  Y2K  system  certified,  there  appears  to  be  an  over  confidence  that 
everything  will  operate  satisfactorily  in  the  Year  2000,  a  situation  decreasing  the 
perceived  importance  of  end-to-end  testing  and  code  scanning.”  The  Air  Force 
stated  that  the  DCPDS  was  a  batch  processing  system  and  that  it  participated  in 
the  end-to-end  test  in  the  same  manner  that  the  system  operated.  The  only 
difference  was  the  amount  of  time  that  the  data  was  saved.  In  normal 
operations,  data  would  be  saved  for  up  to  2  weeks  before  being  transmitted. 

The  action  officer  for  the  USD(P&R)  agreed  with  the  Air  Force  that  the  DCPDS 
had  been  sufficiently  tested  and  did  not  require  the  Air  Force  to  do  further  tests 
of  the  DCPDS. 

A  more  rigorous  end-to-end  test  would  have  provided  DoD  with  additional 
assurances  that  the  DCPDS  would  continue  to  operate  without  interference  from 
year  2000  problems.  Because  of  the  remaining  risk,  the  effectiveness  of  the 
contingency  plan  for  the  DCPDS  and  the  results  of  exercising  the  contingency 
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plan  become  increasingly  important.  As  previously  stated,  the  Inspector 
General,  DoD,  will  perform  an  independent  review  of  the  effectiveness  of 
Y2K  operational  contingency  plans  for  DoD  personnel  systems. 

Planning  and  Executing  the  End-to-End  Tests 


The  Master  Plan  generally  did  not  provide  the  Military  Services  with  specific 
guidance  on  how  to  conduct  end-to-end  testing.  The  details  were  left  to  the 
Military  Services  to  incorporate  into  their  Service-level  test  plan.  Therefore, 
the  Master  Plan  did  not  provide  a  standard  methodology  for  the  Military 
Services  to  follow  in  documenting  the  procedures,  status,  and  results  of  their 
end-to-end  tests.  Consequently,  the  Military  Services  used  various  methods  to 
document  and  track  the  testing  and  results.  For  example,  the  Army  established 
extensive  procedures  for  recording,  verifying,  and  reporting  the  test  results  for 
each  system  in  the  thin  threads.  The  Army  also  established  a  test  operations 
center,  which  monitored  the  test  results  and  maintained  a  detailed  accounting  of 
the  end-to-end  test.  The  test  operations  center  also  had  onsite  representatives  at 
each  test  to  assist  in  monitoring  the  test  and  ensuring  the  necessary 
documentation  was  collected. 

The  Navy  test  plan,  if  followed,  addresses  the  required  elements.  The  test  plan 
describes  the  critical  functions,  interfaces,  test  scenarios,  and  has  procedures  for 
test  execution,  data  analysis,  and  error  recovery.  Data  flow  diagrams  and 
interface  listings  were  descriptive  and  helpful. 

The  USD(P&R)  made  the  observation  that  the  Air  Force  test  plan  was  well 
written  and  provided  a  framework  for  Y2K  testing.  However,  USD(P&R)  was 
concerned  that  the  Air  Force  was  not  following  the  procedures  in  its  own  test 
plan  and  that  documentation  was  generally  not  available.  USD(P&R)  stated  that 
it  appeared  testing  was  being  done  in  an  “ad  hoc”  manner,  and  test  scenarios 
were  being  developed  as  the  tests  were  being  conducted.  Specifically,  the 
contractor  hired  by  the  Office  of  the  USD(P&R)  concluded  in  its  preliminary 
assessment  during  the  end-to-end  testing  of  Air  Force  critical  personnel  systems 
that: 

•  only  a  few  test  scenarios  existed  and  the  test  documentation  was 
incomplete  or  did  not  exist,  and 

•  the  Air  Force  was  not  aware  of  the  effort  required  to  properly 
conduct  and  document  Y2K  end-to-end  testing  in  order  to  satisfy  due 
diligence  requirements. 

The  Marine  Corps  had  not  developed  an  overall  end-to-end  test  plan  to  provide 
overall  guidance  and  direction  on  testing  its  personnel  systems  and  threads.  The 
Marine  Corps  was  developing  its  overall  plan,  even  though  it  had  almost 
completed  testing  of  the  eight  functional  areas  defined  by  USD(P&R).  Instead 
of  developing  an  overall  end-to-end  test  plan,  the  Marine  Corps  drafted  a  plan 
that  addresses  the  functions  that  remained  to  be  tested.  Consequently,  for  the 
tests  conducted,  the  Marine  Corps  was  not  consistent  in  documenting  the  test 
scenarios,  test  results,  or  dates  tested.  For  example,  the  report  prepared  by  the 
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contractor  for  the  end-to-end  test  conducted  on  the  Marine  Corps’  mobilization 
thread  did  not  provide  any  details  on  the  test  results.  The  report  stated  only  that 
no  problem  had  been  encountered  during  the  testing.  We  did  not  make  a 
recommendation  addressing  the  development  of  a  standard  methodology  for 
documenting  the  test  procedures,  status,  and  results  because  the  Army, 

Air  Force,  and  Marine  Corps  had  nearly  completed  their  end-to-end  testing. 

Testing  of  Critical  Dates.  The  Master  Plan  states  that  three  calendar  date 
periods  specifically  affect  personnel  processing  and  must  be  tested.  The  three 
calendar  date  periods  are  the  change  over  dates  of  December  31,  1999,  to 
January  1,  2000;  February  28,  2000,  to  February  29,  2000;  and  February  29, 
2000,  to  March  1,  2000.  The  Master  Plan  further  stated  that  the  Commanders 
in  Chief  operational  evaluations  had  already  found  failures  for  all  three  of  those 
time  periods  in  their  testing.  Even  though  the  USD(P&R)  made  the  assertion 
that  the  dates  must  be  tested  in  the  end-to-end  test  of  personnel  systems,  the 
Army  and  the  Marine  Corps  did  not  include  all  the  critical  test  dates  in  their  test 
efforts.  The  Army  tested  two  of  the  three  required  date  periods:  December  31 , 
1999,  to  January  1,  2000;  and  February  28,  2000,  to  Febmary  29,  2000.  At  an 
April  30,  1999,  planning  meeting  we  attended,  the  Army  indicated  that  it  would 
only  test  two  of  the  date  periods  considered  critical  in  the  Master  Plan. 
Representatives  of  the  USD(P&R)  did  not  object  to  the  Army  not  testing  the 
February  29,  2000,  to  March  1,  2000.  The  action  officer  for  the  USD(P&R) 
stated  that  because  the  Army  had  tested  the  calendar  year  and  leap  year  change 
over  date  periods,  the  USD(P&R)  was  not  concerned  that  the  Army  did  not  test 
the  February  29,  2000,  to  March  1,  2000,  date  period  required  in  the  Master 
Plan.  The  Marine  Corps  did  not  include  the  leap  year  date  change  over  period 
in  its  end-to-end  testing  of  the  mobilization  thread.  At  the  instruction  of  the 
USD(P&R),  the  Marine  Corps  successfully  retested  the  mobilization  thread  to 
include  the  leap  year  change  over  date  period.  The  Air  Force  tested  all  three 
critical  date  changes  and  the  Navy  plans  to  test  the  three  required  date  periods. 
We  did  not  make  a  recommendation  addressing  the  date  that  the  Army  and 
Marine  Corps  did  not  test  because  the  Under  Secretary  of  Defense  for  Personnel 
and  Readiness,  subsequent  to  development  of  the  Master  Plan,  decided  that 
testing  the  February  29,  2000,  to  March  1,  2000,  date  period  was  not  applicable 
in  the  personnel  operational  batch  data  processing  environment. 

Additional  Opportunities  to  Mitigate  Risk 


There  are  additional  methods  that  the  Military  Services  can  to  use  to  reduce  the 
risk  of  Y2K  failure  of  mission-critical  personnel  systems,  especially  those 
systems  that  have  not  been  properly  exercised  in  an  end-to-end  test.  These 
methods  include  reexamining  the  interfaces  between  systems  and  examining 
application  software  through  the  use  of  code  scanning  tools. 

Reexamining  Interfaces.  One  risk  mitigation  method  is  to  reexamine  system 
interface  agreements  and  interface  testing  to  assure  that  date  data  passed 
between  systems  will  work  correctly.  End-to-end  testing  should  ensure  the 
continuity  of  critical  support  functions  and  involve  core  processes,  the  system 
interfaces  required,  and  the  flow  of  data  through  the  interfaces.  The  exchange 
of  data  through  the  interfaces  is  critical  because  they  have  the  potential  to 
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propagate  errors  from  one  core  process  to  another.  If  an  organization  is  unable 
to  participate  in  an  end-to-end  test,  one  way  to  reduce  the  risk  of  losing 
continuity  of  critical  support  functions  is  to  reexamine  the  critical  interfaces  and 
the  data  exchanged  through  them.  Procedures  for  reexamining  interfaces  and 
data  include: 

•  reviewing  the  interface  strategy  with  interface  partners, 

•  using  test  files  to  simulate  the  interfaces, 

•  sharing  test  files  with  interface  partners  so  they  may  conduct  their 
own  simulated  tests,  and 

•  ensuring,  when  possible,  exchanged  year  and  date  fields  are  four 
digits. 

Because  there  is  a  high  risk  that  the  Navy  may  not  complete  or  will  have  to  rush 
through  end-to  end  testing  to  complete  it  in  time,  we  believe  that  the  Navy 
should  perform  the  interface  reexamination  procedures. 

Scanning  Computer  Software.  Another  risk  mitigation  method  is  to  use  Y2K 
analysis  and  renovation  tools  to  examine  application  software  for  mission- 
critical  personnel  systems.  DoD  has  placed  emphasis  on  computer  code  testing 
and  has  purchased  software  for  scanning  software  for  Y2K  errors.  In  an 
August  11,  1999,  memorandum,  the  Assistant  Secretary  of  Defense  (Command, 
Control,  Communications  and  Intelligence)  strongly  recommended  that  every 
DoD  activity  perform  software  maintenance  using  automated  software  quality 
and  testing  tools  to  verify  the  integrity  of  remediated  code.  Some  DoD 
organizations  had  already  planned  to  take  the  initiative  of  scanning  software 
code  to  provide  additional  assurance  that  the  Y2K  risk  is  reduced.  The  Y2K 
action  officer  for  the  Army  Deputy  Chief  of  Staff  for  Personnel  stated  that  the 
Army  may  perform  computer  code  scanning  on  its  personnel  systems.  The  Air 
Force  has  plans  to  scan  its  mission-critical  personnel  and  finance  systems.  The 
Navy  and  the  Marine  Corps  do  not  plan  to  perform  computer  code  scanning  on 
their  systems.  The  Inspector  General,  DoD,  believes  that  code  scanning  should 
be  used  aggressively  to  provide  additional  assurance  that  mission-critical 
systems  will  be  operational  in  Y2K  and  beyond,  especially  when  any  doubt 
exists  about  the  executability  or  rigor  of  the  end-to-end  tests.  It  is  advisable  that 
the  Navy  use  a  code  scanning  tool  to  verify  that  all  mission-critical  personnel 
systems’  application  software  is  Y2K  compliant  since  these  systems  may  not 
complete  end-to-end  testing. 


Conclusion 


The  USD(P&R)  and  the  Military  Services  are  still  conducting,  evaluating,  and 
reporting  on  end-to-end  tests  for  mission-critical  personnel  systems.  It  is  vitally 
important  that  USD(P&R)  and  the  Military  Services  continue  efforts  to  provide 
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DoD  with  a  level  of  confidence  that  all  mission-critical  personnel  systems  will 
demonstrate  operational  readiness  in  Y2K.  Of  equal  importance  is  the  need  to 
take  action  to  mitigate  the  risk  of  the: 

•  Navy  not  completing  end-to-end  testing  of  its  personnel  systems,  and 

•  Air  Force  not  subjecting  the  DCPDS  to  a  higher  level  test. 

There  is  sufficient  remaining  risk  to  warrant  the  Navy  and  Air  Force  taking 
additional  risk  mitigation  measures.  Particular  attention  is  needed  to  make  sure 
that  contingency  plans  are  in  effect  and  that  the  contingency  plans  have  been 
exercised.  In  addition,  the  Navy  should  examine  100  percent  of  the  application 
software,  through  the  use  of  code  scanning  tools,  and  use  interface 
reexamination  procedures  for  all  mission-critical  systems  in  its  personnel 
functional  area. 

Recommendation  and  Management  Comments 


We  recommend  that  the  Deputy  Chief  of  Naval  Personnel  require  that 
interface  reexamination  procedures  be  performed  for  all  Navy  mission- 
critical  personnel  systems  and  that  advanced  automated  scanning  tools  be 
used  to  examine  all  of  the  application  software  for  those  systems. 

Navy  Comments.  The  Chief  of  Naval  Personnel,  Bureau  of  Naval  Personnel, 
concurred  and  stated  that  the  Navy  has  been  continually  reviewing  the  interface 
strategy  with  its  interface  partners  and  simulating  the  interfaces  thoroughly 
during  the  end-to-end  testing.  The  Navy  is  also  using  advanced  automated 
scanning  tools  to  examine  the  application  software  in  the  mission-critical 
personnel  systems.  The  Navy  is  confident  that  all  end-to-end  testing  will  be 
completed  by  the  end  of  October  1999,  and  that  its  personnel  systems  will 
operate  beyond  January  1,  2000.  For  the  full  text  of  Navy  comments,  see  the 
Management  Comments  section  of  the  report. 

Additional  Management  Comments  on  the  Finding  and  Audit 
Responses 


Although  the  report  did  not  make  specific  recommendations  to  the  Air  Force 
and  the  Defense  Information  Systems  Agency,  they  provided  comments  on  the 
finding.  For  the  full  text  of  Air  Force  and  the  Defense  Information  Systems 
Agency  comments,  see  the  Management  Comments  section  of  the  report. 

Air  Force  Comments.  The  Assistant  Director,  Communications  and 
Information,  Air  Force  Communications  and  Information  Center,  disagreed  with 
assertions  in  the  report  regarding  the  effectiveness  of  the  end-to-end  test  of  the 
DCPDS.  The  Assistant  Director  provided  extensive  detail  concerning  the 
comprehensive  Y2K  test  that  the  Air  Force  conducted  on  the  interface  between 
the  Defense  Civilian  Personnel  Data  System  and  the  Defense  Civilian  Pay 
System.  The  Assistant  Director  also  stated  the  report  incorrectly  asserted  that 
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the  sole  reason  for  conducting  the  test  using  the  saved  test  data  was  to  avoid 
having  to  replicate  the  operational  hardware  to  perform  the  end-to-end  test  of 
the  system.  Although  there  was  an  economic  benefit,  using  the  saved  data 
permitted  the  Air  Force  to  replicate  the  operation  of  the  DCPDS  just  as  it  is 
used  in  the  field  and  allowed  more  personnel  transactions  that  affect  pay  to  be 
processed  and  tested. 

Audit  Response.  Although  the  DCPDS  completed  the  system-level  test,  we 
maintain  that  it  was  not  exercised  in  a  separate,  higher  level  test  as  required  by 
the  Defense  Management  Plan.  We  agree  with  the  Assistant  Director  that  the 
data  output  from  DCPDS,  the  interface  between  the  DCPDS  and  the  Defense 
Civilian  Pay  System,  and  the  Defense  Civilian  Pay  System  itself  were  properly 
tested  in  the  end-to-end  test,  but  the  DCPDS  was  not.  Concerning  the  additional 
reasons  that  the  Air  Force  provided  for  using  the  saved  data,  we  have  modified 
the  report  to  reflect  those  additional  reasons. 

Defense  Information  Systems  Agency  Comments.  The  Inspector  General, 
Defense  Information  Systems  Agency,  disagreed  with  statements  in  the  report 
that  the  Navy  was  late  in  starting  its  end-to-end  testing  because  the  Defense 
Information  System  Agency  was  unable  to  provide  the  necessary  platforms, 
manpower,  and  services  to  test  the  systems.  The  Inspector  General  stated  that 
there  were  several  reasons  for  the  late  start  in  the  Navy  testing  and  provided 
information  describing  the  reasons. 

Audit  Response.  We  modified  the  report  to  include  the  information  that  the 
Inspector  General  provided  concerning  the  testing  delays. 
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Appendix  A.  Audit  Process 


This  report  is  one  in  a  series  of  reports  being  issued  by  the  Inspector  General, 
DoD,  in  accordance  with  an  informal  partnership  with  the  DoD  Chief 
Information  Officer  to  monitor  DoD  efforts  to  address  the  year  2000  computing 
challenge.  For  a  list  of  audit  projects  addressing  the  issue,  see  the  year  2000 
web  page  on  the  IGnet  at  http://www.ignet.gov. 

Scope 


We  reviewed  documentation  dated  June  1998  to  August  1999.  The 
documentation  included  policies  and  procedures  issued  by  the  USD(P&R)  and 
established  for  the  end-to-end  testing  of  DoD  personnel  Sanctions.  We  reviewed 
and  analyzed  the  Military  Services  end-to-end  test  plans  and  test  results  for 
personnel  functions.  We  also  reviewed  the  USD(P&R)  initial  assessments  of  the 
Military  Services  test  plans  and  test  results. 

DoD-wide  Corporate  Level  Government  Performance  and  Results  Act 
Goals.  In  response  to  the  Government  Performance  Results  Act,  the 
Department  of  Defense  has  established  2  DoD-wide  goals  and  7  subordinate 
performance  goals.  This  report  pertains  to  achievement  of  the  following  goal 
and  subordinate  performance  goal: 

Goal  2:  Prepare  now  for  an  uncertain  future  by  pursuing  a  focused 
modernization  effort  that  maintains  U.S.  qualitative  superiority  in  key 
warfighting  capabilities.  Transform  the  force  by  exploiting  the 
Revolution  in  Military  Affairs,  and  reengineer  the  Department  to  achieve 
a  21st  century  infrastructure.  Performance  Goal  2.2:  Transform  U.S. 
military  forces  for  the  future.  (OO-DoD-2.2) 

DoD  Functional  Area  Reform  Goals.  Most  major  DoD  functional  areas  have 
also  established  performance  improvement  reform  objectives  and  goals.  This 
report  pertains  to  achievement  of  the  following  functional  area  objectives  and 
goals: 

•  Information  Management  Functional  Area.  Objective:  Become  a 
mission  partner.  Goal:  Serve  mission  information  users  as  customer. 
(ITM-1.2) 

•  Information  Management  Functional  Area.  Objective:  Provide 
services  that  satisfy  customer  information  needs.  Goal:  Modernize 
and  integrate  Defense  information  infrastructure.  (ITM-2.2) 

•  Information  Management  Functional  Area.  Objective:  Provide 
services  that  satisfy  customer  information  needs.  Goal:  Upgrade 
technology  base.  (ITM-2.3) 
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General  Accounting  Office  High-Risk  Area.  The  General  Accounting  Office 
has  identified  several  high-risk  areas  in  the  DoD.  This  report  provides  coverage 
of  the  Information  Management  and  Technology  high-risk  area. 


Methodology 

Audit  Type,  Dates,  and  Standards.  We  performed  this  program  audit  from 
April  1999  through  July  1999,  in  accordance  with  auditing  standards  issued  by 
the  Comptroller  General  of  the  United  States,  as  implemented  by  the  Inspector 
General,  DoD. 

Use  of  Technical  Assistance.  We  received  technical  assistance  from  the 
Computer  Engineers  in  the  Inspector  General,  DoD,  Audit  Followup  and 
Technical  Support  Directorate  to  obtain  assistance  with  reviewing  the 
sufficiency  of  test  plan  and  test  results. 

Use  of  Computer-Processes  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  DoD.  Further  details  are  available  on  request. 

Management  Control  Program.  We  did  not  review  the  management  control 
program  related  to  the  overall  audit  objective  because  DoD  recognized  the 
year  2000  computing  problem  as  a  material  management  control  weakness  in  the 
FY  1998  Annual  Statement  of  Assurance. 

Summary  of  Prior  Coverage 


The  General  Accounting  Office  and  the  Inspector  General,  DoD,  have 
conducted  multiple  reviews  related  to  year  2000  issues.  General  Accounting 
Office  reports  can  be  accessed  over  the  Internet  at  http://www.gao.gov. 
Inspector  General,  DoD,  reports  can  be  accessed  at  http://www.dodig.osd.mil. 
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Appendix  B.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Under  Secretary  of  Defense  (Personnel  and  Readiness) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence) 
Deputy  Chief  Information  Office  and  Deputy  Assistant  Secretary  of  Defense  (Chief 
Information  Officer  Policy  and  Implementation) 

Principal  Director  for  Year  2000 
Director,  Defense  Logistics  Studies  Information  Exchange 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Army 
Inspector  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 

Deputy  Chief  of  Naval  Personnel 

Auditor  General,  Department  of  the  Navy 

Inspector  General,  Department  of  the  Navy 

Inspector  General,  Marine  Corps 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 

Deputy  Chief  of  Staff  for  Personnel 

Auditor  General,  Department  of  the  Air  Force 

Inspector  General,  Department  of  the  Air  Force 
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Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Information  Systems  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 

Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 

Office  of  Information  and  Regulatory  Affairs 
General  Accounting  Office 

National  Security  and  International  Affairs  Division 
Technical  Information  Center 

Director,  Defense  Information  and  Financial  Management  Systems,  Accounting  and 
Information  Management  Division 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Committee  on  Governmental  Affairs 

Senate  Special  Committee  on  the  Year  2000  Technology  Problem 

House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Reform 

House  Subcommittee  on  Government  Management,  Information,  and  Technology, 
Committee  on  Government  Reform 

House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  Technology,  Committee  on  Science 
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Bureau  of  Naval  Personnel  Comments 


DEPARTMENT  OF  THE  NAVY 

BUREAU  OP  NAVAL  PERSONNEL 
5725  INTEGRITY  DRIVE 
MILLINGTON  TN  53955-0000 


IN  REPLY  REFER  TO 
7540 

Ser  00*0/1999240 
0-1  Oct  99 


From:  Chief  of  Naval  Personnel 
To:  DDD  Inspector  General 

Subj:  DRAFT  AUDIT  REPORT  ON  END-TO-END  TESTING  FOR  PERS OMNEJ.  SYSTEMS 
(PROJECT  NO.  9AL-0098) 

Ref:  (a)  Subject  DODIG  ltr  Of  10  Sep  55 

Enel:  (1)  Response  to  subject  draft  report 

1.  Per  reference  (a) ,  our  response  is  provided  ir  enclosure  fl).  If 
you  have  any  questions,  the  POC  in  this  office  is  Mr-  Richard 
Cosgrove  at  Coraro  901-974-3034  (DSN  882)  . 


Copy  to: 

ASN  {>15  RA) 

EKO-31 
DON  CIO 
CHNAVPERS 
CNO  (NIB) 

NAVFFRSCOM  (PERS-OO,  07) 
NRISO 


D.M.  CASHGAUGH 
Ey  direction 


17 


Navy  Comment $ 
on 

OAIGtA)  Draft  Report  on  10  September  1939 
On 

Bnd-to-End  Testing  for  Personnel  systems 
Project  tfo.  9hX.-003S 

£mnr\ary  of  oaig(A)  Findings,.  Conclusions  and  Rocon-mandationg 

^By  not  be-ing  able  to  $t$rt  its  end-to-end  testing  until  September 
1999/  the  Navy  may  not  have  sufficient  time  to  complete  the  testing, 
evaluate  the  test  results,  resolve  y2k  issues  that  might  be 
encountered,  and  retest,  if  necessary." 

Summary  of  the  Navy  Position 

Navy  started  End-to-End  testing  on  A  September  129$  completing 
active  duty  testing  with  no-  Y2K  related  errors  on  20  September  1999, 
and  is  on  schedule  to  complete  all  testing  by  the  end  of  October 
1999.  Navy  is  confident  that  all  end-to-end  testing  will  be 
successfully  completed  within  the  time  constraints  and  personnel 
systems  will  operate  beyond  l  January  2000. 

QM6 (A)  Re comroendat ion :  *We  recommend  that  the  Deputy  Chief  Of  Staff:  of 
Navy  Personnel  require  that  interface  reexamination  procedures  be 
performed  for  dll  Navy  mission- critical  personnel  systems  -and  that 
advanced  automated  scanning  tools  be  used  to  examine  LOO  percent  of  thfe 
application  software  for  those  systems." 

Navy  Manafrement  Responses  concur. 

1.  Cede  Scanning  -  Navy  mission-critical  personnel  systems  are  being 
examined  with  the  advance  automated  scanning  tools. 

2.  Interface  Testing  - 

a)  The  N&vy  has  completed  all  of  the  active  duty  E2B  test  strings 

(accession,  retirement,  locate,  mobilization  active  duty  gain,  and 
deployment )  with  no  Y2K  related  errors  discovered.  Pay  transactions 
from  testing  were  passed  to  DPAS  on  17  September  1999.  Results  axe 
pending. 

b}  Navy  Personnel  systems  E2E  remain  on  schedule-  casualty  Kanacaticnt 
Testing  is  scheduled  for  2  October  1339.  Fo] loved  by  05D  Reserve 
E2E  testing  scheduled  for  the  week  of  11  October  1953. 
c)  The  procedure  listed  for  reexamining  interfaces  is  extremely  time- 
consuming,  manpower  intensive,  and  requires  a  massive  coordination 
effort.  Incremental  interface  testing  which  was  completed  was  also- 
considered  as  a  strategy  instead  of  OSD  B2Z  testing.  However,  after 
careful  review,  OSD  (PtR)  considered  it  a  non-viablc  option  to 
replace  E2E  testing. 


Enel  Hi 
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d)  Specific  comments  regarding  interface  testing  procedure  listed  in. 
the  draft  report  are  addressed  below: 

(1)  The  system  interface  approach  referred  to  in  part  one  has  been 
tested  continually. 

(2)  Parts  2  and  3  of  the  procedure  are  component  parts  of  E2E 
testing  and  are  being  thoroughly  exorcised  during  Navy  E2E 
testing, 

<3}  Ensuring  four-digit  year  fields  (part  Jj)  as  part  of  the  basic 
strategy  employed,  in  the  renovation  of  code.  The  Navy 
adopted  the  windowing  techniques  that  support  the  two-digit 
year  fields  to  enable  completion  of  the  code  renovation  in  a 
xinely  manner,  where  practical/  four  digit  field  years  were 
incorporated , 

OAIG  Comraentg  from  Page  6: 

VSD(P&H)  ha*  expressed  concerns  regarding  the  Navy  ability  to 

aomplete  end-to-end  testing  of  its  personnel  systems  within  the  timeframe 
established.  The  U3D(P£R)  concerns  related  to  the  possible  lack  of  Navy 
financial  xe sources  to  accomplish  the  cequired  testing  and  lack  of  the 
Bureau  of  Naval  Personnel  system  documentation  and  institutional 
knowledge. w 


Through  intense  scrutiny  and  reprogratming,  funding  has  been  identified  to 
accomplish  the  required  testing,  internal  reorganizations  within  supers 
and  the  BRAc  move  to  Millington  did  reduce  the  numbers  of  valuable  people 
with  in-depth  business  knowledge  of  the  tfavy  personnel  systems.  Together 
BUPERS,  NftJSO  and  supporting  contractors  developed  training  plans,-  formed 
teams  of  functicnal/technical  personnel  to  acquire  the  in-depth  knowledge 
required  to  renovatt,  certify  and  test  Navy  personnel  systems.  Much  of 
the  business  and  process  knowledge  has  been  regained  end  Navy  E2E  testing 
is  on  schedule  for  completion  by  31  October. 

0A1G  Comment a  from  Page  6: 

M Additional  risk  mitigation  measures  are  needed  and  the  USD(PSR)  needs  to 
be  aware  of  the  results  of  the  actions  taken.  One  such  measure  would  be 
an  independent  review  of  the  contingency  plans  and  the  results  of 
exercising  the-  contingency  plans  for  the  personnel  systems  in  the  Navy .  * 

Response: 

fiavy  has  developed  a  draft  Operational  CCOP  that  supports  the  OSD  defined 
mission  critical  areas.  This  plan  was  briefed  to  DOD  IG  on  20  September 
19S$  as  part  of  their  review  of  Coop  and  contingency  plans  in  support  of 
X2K.  Navy  intends  to  hold  a  roundtable  test  of  all  components  of  the  OCOP 
by  1  November  1999.  Results  and  updates  from  tests  will  be  repotted  o.S 
required. 


Enel  (1) 
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Department  of  the  Air  Force 
Comments 


DEPARTMENT  OFTHE  AIR  FORCE 

HEADQUARTERS  AIR  FORCE  COMMUNICATIONS  ftUO 
INFORMATION  CENTER 


l  S  StP  '.sss 

MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING 
OFFICE  OF  THE  INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 

FROM:  HQ  USAF/SC 

1250  Air  Force  Pentagon 
Washing  on  DC  20330- 1250 

SUBJECT:  Draft  Audit  Report  on  End-to-End  Testing  for  Personnel  Systems.  10  Sep  1 999 
(Project  No  9AL-009&) 

This  is  in  rcpl)  Jo  your  memorandum  requesting  the  Assistant  Secretary  of  the  Air  Force 
(Financial  Management  and  Comptroller)  to  provide  Air  Force  comments  on  subject  report 

While  you  did  not  make  specific  recommendations  for  the  AF.  we  disagree  with  your 
issejiions  regarding  the  effectiveness  of  the  end-to-end  tested  (he  Defense  Civilian  Personnel 
Data  System  Air  Force  comments  are  attached  Our  point  of  contact  is  Maj  Callahan.  Air  Force 
Year  2000  Office  (AFY2KO).  lie  can  be  irachcd  at  DSN  333-2225  or  703/602-2225. 

ANTHONY  W.  UtLL,  JR  ,  Brig  Gen,  USAF 
Assistant  Director.  Communication  s  and  Information 


Aiiadirnent; 

Air  Force  Comments 


cc: 

AF  CIO 
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Air  Force  Comments  On  Draft  DoDfG  Audit  Report 
(Project  No.  9AL-OOW,  End-U»-End  T^lin*;  for  Personnel  Systems 

GENERAL  AF  COMMENTS.  The  report  focuses  on  testing  performed  between  5  and 
2S  May  1999  and  does  not  accurately  reflect  the  facts  concerning  Ihe  Y2K  end-to-end  test 
conducted  by  the  Air  Force  Personnel  Center  (AFPC).  Testing  occurred  from  5  May 
through  17  .Sep  1999  to  ensure  all  critical  functional  are&xftransaciions  were  fully  tested 
To  duic,  more  than  500.000  of  642.000  tines  of  AF  personnel  system's  axle  have  been 
scanned  without  identifying  a  single,  confirmed  problem  that  would  have  prevented  AF 
personnel  .systems  from  operating  in  ihe  year  2000 

SPbXIFK  .COMMENT 

1. DoDIG.  Executive  Summary,  “Results",  paragraph  2.  ‘The  .Air  Force  end-to-end  test 
involving  the  Defense  Civilian  Personnel  Data  System  was  not  rigorous  by  the  criteria  set 
forth  in  DOD  Year  2000  Management  plan 11 

2.  DoDIG.  Page  3,  "End-to-End  Testing  of  Personnel  Systems”,  paragraph  1,  “Also,  the 
Air  Force  end-lO-end.  test  involving  the  Defense  Civilian  'Personnel  Data  System  v,us  not 
as  rigorous  as  required  by  the  criteria  set  forth  in  DOD  Management  Plan  Instead  the 
Air  Force  chose  io  use  test  data  that  k  saved  from  the  system  certification  test  of  the 
Defense  Civilian  Personnel  System  As  a  result,  additional  risk,  mitigation  procedures 
such  as  independent  reviews  of  contingency  plans,  reexamination  of  interfaces,  and  the 
use  of  code  scanning  tools  on  application  software  arc  warranted  " 

3  DoDIG,  Page  7,  "Use  of  System  Level  Test  Results  for  End-to-End  Testing", 
paragraph  1.  ‘The  Ajr  Force  end-to-end  lest  involving  the  DCPDS  was  not  as  rigorous  as 
required  by  ihe  criteria  .set  forth  in  DOD  Management  Plan  11  “The  DCPDS  did  compile 
ihe  system  level  tcsl  but  was  not  exercised  in  a  separate.  higher  level  test  as  required  by 
the  Defense  Management  Plan  *'  “Instead  of  including  DCPDS  in  the  end-to-end  test,  the 
Air  Force  chose  to  use  test  data  that  it  saved  from  the  system  certification  lest  of  ihe 
DCPDS  for  its  end-to-end  testing-**  ‘The:  ttofa  from  the  DCPPS  was  properly  passed  to 
other  systems:  therefore,  the  interfaces  between  those  systems  were  properly  tested. 
However,  the  Air  Force  did  not  retest  input  and  processing  by  the  DCPDS  during  I  he 
end-to-end  test.”  ‘The  participation  of  the  DCPDS  in  the  end-to-end  test  merely  consisted 
of  the  DCPDS  passing  the  saved  data  to  the  Defense  Civilian  Pay  System  ” 

AF  COMMENTS.  Nonconcur.  The  report  alleges  that  the  AF  Y2K  interface  test 
between  the  Defen.se  Civilian  Personnel  Data  System  (DCPDS)  and  the  Defense  Civilian 
Pay  System  (DCFS')  was  not  as  rigorous  as  Ihw  required  by  the  criteria  set  forth  in  the 
DoD  Management  Plan.  The  Defense  Civilian  Personnel  Management  Service  (CPMSj 
conducted  acomprd^nsive  Y2K  interface  test  ihat  consisted  of  using  test  data  generated 
from  processing  virtually  all  of  the  several  hundred  types  of  personnel  actions  approved 
by  the  Federal  Office  of  Personnel  Management  The  text  datt  w**  generated  during  the 
Y2K  Personnel  functional  test  of  the  legacy  DCPDS  conducted  from  November  to 
December  199K.  The  lest  data  corresponded  to  approximately  2600  different 
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Final  Report 
Reference 


Revised 


combinations  of  personnel  actions  representing  several  DoD  components.  DCPDS  testers 
saved  this  data  in  the  same  manner  replicating  field  operations.  The  lest  <kia  was  ihcn 
transmitted  from  DCPDS  lo  DCPS  The  DCPDS  output  test  data  was  not  altered  in  any 
manner  before  it  was  transmitted  to  DCPS.  DCPS  validated  that  test  data,  created  its 
history  file/)  from  the  first  batch,  and  then  processed  Ihe  test  daU  in  the  same  manner  as  in 
actual  operations  DCPS  then  transmitted  the  reverse,  reject,  and  reconciliation  files 
covering  each  of  the  critical  pay  periods  in  hatch  im>de  bacV  to  DCPDS  for  validation  and 
farther  processing  in  the  same  manner  replicating  field  operations.  The  report 
acknowledges  that  a  USD(P<fcR)  action  officer  ‘'agreed  wuh  the  AF  thai  the  DCPDS  had 
been  sufficiently  tested  and  did  nor  require  the  Air  Force  to  do  further  rests  of  rhe 
DCPDS.” 

All  critical  Y2K  midnight  crossing  dates  encompassing  all  pay  periods  from  late 
December  1999  through  April  2000  were  used  in  the  personnel  actions  that  were 
processed  for  the  Y2K  interface  test  The  Y2K  end-to-end  test  described  above  was 
comprehensive  and  exceeded  the  rctmiremenls  set  forth  in  the  DoD  Management  Plan. 
Plans  were  closely  followed  for  the  Y2K  interface  test  between  the  DCPDS  and  DCPS 
that  was  based  on  the  DoD  Management  Plan 

4  DoDIG.  Page  7,  “Use  of  System  Level  Test  Results  for  End-to-End  I  csting”, 
paragraph  1.  "The  Air  Force  stated  that  by  using  the  saved  data  it  was  able  to  save  $2.3 
million  by  eliminating  the  need  lo  replicate  the  DCPDS  hardware  for  the  cmMo-end  test * 

AF  COMMENTS.  Nonconcur.  The  report  incorrectly  asserts  that  our  sole  reason  for 
conducting  the  Y2K  endku-eml  test  in  this  manner  was  u>  uvrid  having  to  replicate  the 
operational  hardware  to  perform  the  end-to-end  test  of  the  systems  While  there  was  an 
economic  benefit  to  using  saved  test  data,  it  was  not  the  sole  or  primary  reason  for  our 
use  of  saved  test  dai a  Saved  test  data  was  used  to  -replicate  the  operation  of  (he  DCPDS 
just  w  it  is  used  in  the  field.  Use  of  the  test  dau  allowed  more  transactions  to  be 
processed  thereby  testing  more  personnel  transactions  that  affect  pay  Test  procedure* 
were  completely  justified  on  the  basis  that  any  variation  of  the  DCPDS  operation  during 
this  test  would  compromise  ihc  correct  manner  of  conducting  the  Y2Kcndrio»end  test 
between  the  DCPDS  and  DCPS  In  actual  field  operations,  the  output  data  generated  by 
the  personnel  actions  processed  with  the  DCPDS  must  be  saved  by  the  personnel  office, 
and  then  transmitted  in  batch  mode  lo  DCPS  for  subsequent  payroll  actions  effective 
within  the  current  pay  period.  'I  his  data  is  not  transmitted  piecemeal  to  DCPS  far  each 
peisoTUtel  action  processed;  therefore,  all  action*  are  batched  and  flowed  to  DCPS. 

5.  DoDIG.  Page  8.  “Planning  and  Executing  ihe  End-to-End  Tests1',  paragraph  3,  ‘The 
USDfP&R)  made  ihc  observation  that  the  Air  Force  lest  plan  was  well  written  and 
provided  a  framework  for  Y2K  testing  However,  U£D(P&.R)  was  concerned  iha|  the  Air 
Foret  was  not  following  the  procedures  in  ib  own  test  plan  and  that  documentation  was 
generally  not  available  ” 

AF  COMMENTS:  Nonconcur  {Military  Personnel  System)  The  test  plan  lemaitied  in 
dj  ail  pmpo.se  ly  to  allow  the  moruh  of  May  '99  for  internal  evaluation  of  the  limited  test 
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effort  and  determination  of  the  most  appropriate  forms  of  documentation  for  the  record 
Modified  rest  procedures  in  June  included  standardised  test  tools  and  procedures  that 
were  followed  for  the  remainder  of  the  test 

6  DoJ>IG.  Page  S.  “Planning  and  Executing  the  End-to-Fml  Tests 1 paragraph  3 
"USDfP&R)  stated  that  it  appeared  testing  was  being  done  in  an  "ad  hoc*'  manner,  and 
te^t  scenarios  were  being  developed  as  the  tests  were  being  conducted  " 

AF  COMMENTS:  Nonconcur  (Military*  Personnel  System).  The  test  plan  laud  out  high* 
level  funL-riontil  processes  from  within  which  the  functional  OPRs  were  to  test  All  eight 
critical  functions  within  Lhc  personnel  area  identified  by  USD(P&R)  were  used  in  the 
test:  Access,  Mobilize.  Deploy,  Ijocate,  Pay.  Separate.  Retire,  and  Casualty  Support. 

Each  furctional  area  had  a  finite  number  of  scenario  options  available  Assignments,  for 
example,  could  initiate,  change,  or  cancel  an  assignment.  The  daiiy  test  “scenarios"  were 
essentially  limited  to  the  records  themselves  and  Y2 1C  critical  test  dates  associated  with 
each.  Uran$*ciiou.  Each  OFR  presented  their  daily  “scenario1'  from  which  the  Ust 
transactions  were  input.  The  following  duty  day  the  output  was  analyzed  with  the 
appropriate  data  systems  expert  and  certified  by  both  All  necessary  documentation  is 
available  to  support  each  day's  testing. 

7.  LtoDlG,  Page  fi,  "Planning  and  Executing  the  End -to -End  Tests",  paragj  aph  3. 
“Specifically,  the  contractor  hired  by  the  Office  of  the  V'SD(P&R)  concluded  in  jin 
preliminary  assessment  during  the  cnd-lo-cnd  testing:  of  Air  Force  critical  personnel 
systems  that: 

-  only  a  few  test  scenarios  existed  and  the  test  documentation  was  incomplete  or 
did  not  exist 

AF  COMMENT:  Nonconcur.  See  comment  above, 

-  "lhc  Air  forte  was  not  aware  of  the  effort  required  in  properly  conduct  aiul 
document  Y2K end-to-end  testing  in  order  to  satisfy  due  diligence  requirements.” 

AF  COMMENTS:  Nonconcur  (Military  Personnel  System)  The  testing  staff  was  fully 
aware  of  the  effort  required  to  satisfy  the  due  diligence  requirements  and  was  working  to 
that  end  This  draft  audit  report  cites  observations  applying  to  the  first  three  weeks  of  an 
ithmwie  five-month  test  period.  From  June  through  September,  AFPC  lests  included  38 
functional  OPR&  generating  daily  test  transactions  involving  an  average  of  over  1 .600 
individual  transactions,  using  1 10  transaction  types,  within  eight  different  functional 
areas  across  the  military  lifecycle.  Over  1 35,000  total  transactions  were  generated  during 
this  tour-month  test  period  From  IK-20  August,  a  representative  from  U5D(P&R)  and 
his  third-party  contract  auditor  interviewed  our  functional  OFRs  as  well  as  PDS  testing 
staff  members  for  all  testing  accomplished  from  1  June  through  their  visit 

U_  DoDIG.  Page  10  and  1 1  "Conclusion"  “Of  equal  importance  is  the  need  to  Lake 
action  10  mitigate  the  ris^  of  the: 
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-  Air  Force  not  subjecting  (he  DCPDS  to  a  higher  level  test. 

Ah  COMMENTS:  Nonconcur.  I  he  Y 2K  end-io-cnd  ic^i  described  in  this  report  was 
comprehensive  ansi  exceeded  the  requirerwrits  set  forth  in  the  DoD  Management  Flan. 
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Defense  Information  Systems  Agency  Comments 


Why 


Inspector 


DEFENSE  INFORMATION  SYSTEMS  AGENCY 

>0  &  CCL*TI'0VJ3E  no*  a 
ARLIKCTO  VIRGI'M 


General  'll-' 


29  SeptenJoez 


1999 


HSPORTitiDUM  FOR  IX'SPVl'liy:  ULKtt-.AL,  ULRAH7.XKNT  OS  lA-allSt 

(A'i  i'.N  :  ACQUISITION  XhMACilMLNU  L"'  1  RiVi'C-KATt t 


SLfcJLOf:  RESraaSE*  7C;  HO  3  7 G  "SPArT  R7,?riP.T,  “ACX'n  RSrOR*: 

ON  EtsD-70-7.N0  7KS77K'"  FOF  PFA50NNF.L  SYSTEMS 
irROJFr?  #5AT.-o*^a:* 

i-  The  attached  erc.asure  rs  t*£  ci  tidal  FISA  re  spon.H-e  to  tM« 
s-.:rjftct  rlrdr  repr.rr  .  ’"he  H73A  r.nrraenr  s  were  to rwar::*:! 
e.eet ironical- y  cr.  J-J  September  19r9. 


2.  If  yon  have  any  quest ior.s, 
Stein*::,  Audit  Liaisi-n,  at  ; 7 C 


F.nc)  os*-rp  a/s 


please  call  #5 ,  'Iedde  Leu 
3:  ec'^-ent. 


: n.L:i;e:u!  ojt  C:«nfc»idl 


Quality  Information  for  a  Strong  Defense 
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Final  Report 
Reference 


Revised 


Revised 


DiSA  WESTHEM  Response  to  DoD  Inspector  General  Audit  #  9AL-009& 
End-to-End  Testing  for  Navy  Personnel  Systems 

Note  WS$THEM  Comments  aw  informational  vby  Report  contains  no  wcommonfoVons. 

Executive  Summary,  page  i  and  page  3,  Report  statement 

However,  the  Navy  did  oof  plan  to  begin  its  end-to-end  testing  until  September 
1999  because  the  Defense  information  Systems  Agency  was  unable  to  provide 
the  Navy  the  platforms  it  needed  to  test  system  compliance  in  accordance  with 
the  Year  2000  system  certification  process 

And  Page  5,  Report  statement. 

The  delay  in  starting  the  testing  occurred  because  the  Defense  Information 
Systems  Agency  was  unable  tc  provide  the  necessary  platforms,  manpower  and 
services  to  enable  the  Navy  mission-critical  systems  to  be  tested  in  accordance 
with  the  Veer  2000  system  certification  process  Further  complicating  matters, 
the  Defense  Information  Systems  Agency  was  also  contemplating  moving  the 
test  site  from  Chambersburg,  Pennsylvania  to  Columbus,  Ohio 

DISA  WESTHEM  Comments: 

WESTHEM  strongly  non-concurs  with  this  assessment.  These  statements  da  not 
clearly  represent  the  facts  as  they  exist  and  could  lead  to  confusion  on  the  part  of  the 
readership  of  this  document  The  following  information  is  provided  to  clarify  the  facts 

The  Naval  Reserve  Information  Systems  Office  (NRISO)  identified  in  writing  their 
requirements  for  the  support  of  their  Year  20DO  Testing  (Y2K)  in  Sep  98.  This  occurred 
while  Defense  Megaoenter  Chambersburg  {DMC  Chambersburg)  was  in  the  process  of 
converting  all  of  its  customers  to  the  IBM  Operating  System  OS/39C,  Based  on  the 
1997  Quadrennial  Defense  Review,  DMC  Chambersburg  was  also  undergoing  a  down¬ 
sizing  effort.  This  down-sizing  effort  originally  scheduled  all  DMC  Chambersburg 
mainframe  processing  to  be  moved  to  other  Defense  Megacenters  by  Sep  98.  As  a 
result  of  this  down -sizing,  DMC  Chambersburg  began  to  lose  technical  personnel 

Initial  analysis  of  the  requirements  to  support  NRISOs  Y2K  Level  I  testing  identified  the 
approximate  cost  would  be  in  the  9-1 2  roillion-dollar  range.  Upon  review  of  NRISQs 
requirements,  it  was  determined  that  DMC  Chambersburg  lacked  sufficient  hardware 
and  personnel  resources  to  meet  their  need.  NRISO  and  DiSA  personnel  worked  hand- 
in-hand  to  further  refine  the  requirement,  and  as  a  result  were  able  ta  get  the  overall 
cost  for  NRISOs  Y2K  Level  1  testing  down  to  approximately  1,7  million  dollars 
Additionally,  in  order  to  allow  NRtSO  to  meet  its  aggressive  remediation  and  testsing 
schedule,  it  was  also  agree  by  both  DISA  and  NRISO  to  not  migrate  the  personnel 
workload  until  CYOO.  The  costs  and  requirements  for  Level  I  testing  were  identified  and 
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negotiated  with  NRISO  and  finalized  in  a  Memorandum  of  Agreement  (MOA)  between 
NRISO  and  CMC  Chambersburg,  signed  15  Sep  9E 

It  should  be  noted  there  were  no  requirements  for  Level  II  End-to-End  testing  identified 
in  this  MOA 

To  support  NRISO  Level  I  testing,  the  hardware  contract  was  in  plaoe  on  23  Ocl  98  and 
the  equipment  was  delivered  9  Nov  9B.  The  personnel  contract  was  in  place  24  Nov  98, 
While  the  contracting  efforts  were  ongoing,  NRISO  and  DMC  Chambersburg 
coordinated  the  schedule  and  actions  required  to  make  the  system  available  for 
remediation  and  testing  The  two  parties  agreed  to  an  availability  date  of  31  Jan  99, 

This  date  was  met  and  the  system  was  provided  to  NRISO  for  their  testing, 

NRISO  developed  a  schedule  that  called  for  completion  of  all  remediation  wc*K 
application  testing  in  a  rolled-forward  systems  environment  and  subsequent  retrofit  back 
to  their  production  systems  by  1 6  Aug  99  All  of  this  work  was  completed  on  schedule 

During  the  remediation  period  NRISO  identified  the  need  to  conduct  Y2K  Level  II  End- 
to-End  testing  The  new  requirements  for  Level  II  testing  mirrored  the  hardware, 
software,  and  personnel  requirements  that  were  allocated  and  made  available  for  the 
Level  I  remediation  and  testing.  A  new  Level  II  End-to-End  testing  agreement  was 
signed  between  DMC  Chambersburg  and  NRISO  on  27  Apr  99.  However,  EnrMo-Enti 
testing  could  not  commence  until  the  Navy  accepted  as  satisfactory  the  results  of 
NRlSO's  Level  I Y2K  tested  applications  These  applications  were  subsequently 
accepted  and  NRiSO  began  Level  II  End-to-End  testing  on  13  Sep  99 

NRISO  has  successfully  completed  the  Year  End  and  Leap  Year  2000  rollovers  and 
continues  to  meet  planned  testing  schedules  through  the  concerted  efforts  of 
themselves  and  DMC  Chambersburg  The  completion  of  the  current  EncMo-End  Level  11 
testing  is  scheduled  to  end  17  Oct  99.  This  will  provide  the  most  comprehensive 
validation  of  Y2K  compliancy  of  the  Navy  personnel  applications,  OlSA  stands  ready  to 
continue  to  support  the  NRISO  Y2K  validation  efforts  with  the  same  degree  of  efficiency 
and  responsiveness  demonstrated  by  the  accomplishments  to  date. 

The  Inspector  General  Report  statements  annotated  above  leads  the  reader  to  believe 
that  DISA  was  unable  to  support  the  Navy  Y2K  test  requirements.  As  can  be  seen  here 
in  the  description  of  events  DISA  responded  to  and  met  the  late  breaking  and  ever 
changing  requirements  tor  NRISOs  Y2K  Level !  and  Level  II  testing  in  a  timely  manner. 
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